Characterizing Medicine Quality by Active Pharmaceutical Ingredient Levels: A Systematic Review and Meta-Analysis across Low- and Middle-Income Countries

ABSTRACT. Substandard and falsified medicines are often reported jointly, making it difficult to recognize variations in medicine quality. This study characterized medicine quality based on active pharmaceutical ingredient (API) amounts reported among substandard and falsified essential medicines in low- and middle-income countries (LMICs). A systematic review and meta-analysis was conducted using PubMed, supplemented by results from a previous systematic review, and the Medicine Quality Scientific Literature Surveyor. Study quality was assessed using the Medicine Quality Assessment Reporting Guidelines (MEDQUARG). Random-effects models were used to estimate the prevalence of medicines with < 50% API. Among 95,520 medicine samples from 130 studies, 12.4% (95% confidence interval [CI]: 10.2–14.6%) of essential medicines tested in LMICs were considered substandard or falsified, having failed at least one type of quality analysis. We identified 99 studies that reported API content, where 1.8% (95% CI: 0.8–2.8%) of samples reported containing < 50% of stated API. Among all failed samples (N = 9,724), 25.9% (95% CI: 19.3–32.6%) reported having < 80% API. Nearly one in seven (13.8%, 95% CI: 9.0–18.6%) failed samples were likely to be falsified based on reported API amounts of < 50%, whereas the remaining six of seven samples were likely to be substandard. Furthermore, 12.5% (95% CI: 7.7–17.3%) of failed samples reported finding 0% API. Many studies did not present a breakdown of actual API amount of each tested sample. We offer suggested improved guidelines for reporting poor-quality medicines. Consistent data on substandard and falsified medicines and medicine-specific tailored interventions are needed to ensure medicine quality throughout the supply chain.


INTRODUCTION
Poor-quality medicines pose a significant threat to patients and health systems globally because they may be ineffective, resulting in increased length of illness and the need for further treatment. [1][2][3][4][5] In worse cases, poor-quality medicines can cause severe adverse reactions or lack life-saving active ingredients, resulting in avertable deaths. 2,3,6,7 In 2017, the World Health Organization (WHO) adopted formal definitions of substandard and falsified medical products to describe poor-quality medicines. 8 Substandard medicines refer to "authorized medical products that fail to meet either their quality standards or specifications, or both." 6 Falsified medicines are defined as "medical products that deliberately or fraudulently misrepresent their identity, composition, or source." 6 A variety of testing methods can detect substandard and falsified medicines, including visual and physical inspection, dissolution testing, and analysis of active pharmaceutical ingredient (API) content. 3 A WHO review found that 1 in 10 essential medicines in low-and middle-income countries (LMICs) failed tests for quality. 2 Two recent studies estimated a similar range from 13% to 25%. 9,10 However, these analyses report all failed samples together, without distinguishing what pharmacopeia standards are being applied and how much the failed samples deviated from these specifictions. 11 Understanding how much substandard and falsified medicines deviate from pharmacopeial standards for API content would add needed depth to the interpretation of overall prevalence of poor-quality medicines, and has implications for interventions to address them. 11 Although many quality attributes (e.g., disintegration, dissolution, degradation, and presence of impurities) can affect treatment outcomes, the API content of a medicine is highly associated with its therapeutic efficacy and has implications for the development of antimicrobial resistance. 12 Broadly, medicines with insufficient API content reduce therapeutic efficacy and have a more extensive impact on resistance compared with medicines with no API. 12 The extent of deviation in the API content can indicate where supply chain issues are and allow for better tailoring of interventions. For example, some drugs may deviate only slightly from specifications, most likely indicating inadequate manufacturing or poor storage conditions. On the other hand, medicines with substantially low amounts of API, no API, or an incorrect API may indicate fraud, which may be further investigated by the pharmaceutical company or national medicines regulatory authorities (NMRAs). Because manufacturing falsified medicines is criminal, substandard and falsified medicines have different legal ramifications and require distinct solutions. A 2016 report on quality of lifesaving medicines differentiated samples by levels of deviation to understand the therapeutic effects of the products. 13 However, studies differentiating poor-quality medicines by API content levels have not previously been documented.
This systematic review and meta-analysis updates prior analyses 9 and seeks to break down the prevalence of substandard and falsified essential medicines in LMICs by API levels. We examined amounts of API content among essential medicines in studies that tested medicine quality in LMICs. We also offer guidance on how to improve reporting of poor-quality medicines in future medicine quality studies.

MATERIALS AND METHODS
Systematic review. We searched for medicine quality studies in LMICs. First, we used searches from PubMed, EconLit, Global Health, Embase, and Scopus covering publications up to November 3, 2017. 9 Search terms involved iterations of the terms "substandard and falsified medicines," "quality of medicines," and "lowand middle-income countries." Second, we updated this search in PubMed to February 4, 2020. Third, we searched the Infectious Diseases Data Observatory's Medicine Quality Scientific Literature Surveyor, an online platform that gathers medicine quality studies, from inception through September 10, 2020. This database reviews PubMed, Google Scholar, Embase, the WHO, the U.S. Pharmacopeia, Medical Regulatory Agencies' websites, and other sources to include scientific reports on medicine quality in English, French, and Spanish. 14 Further details of the search strategy and search terms are included in the supplemental materials.
Studies were included in the systematic review if they assessed medicine quality, examined essential medicines as classified by the WHO, were conducted in LMICs as classified by the World Bank, and reported the quantity of samples tested and failed. 15 Included studies reported original sampling and testing data where samples were taken or purchased directly from markets. To ensure adequate statistical power and study quality, we included studies that tested a minimum of 50 samples. Studies without primary data, publications without full texts, and case reports were excluded.
After removing duplicates, each publication was independently reviewed for potential inclusion by two of four reviewers (H. C., Y. L., C. H., and T. Y.) based on the title and abstract, followed by a full-text review. Any inconsistencies between dual reviewers were addressed by a third independent reviewer (S.O.). Data abstraction was completed independently by three abstractors (H.C., Y.L., and C.H.). Discrepancies between abstracted results were discussed and resolved between the abstractors and S. O. Study data, including the sample size, type of sampling and testing methods, publication year, country where samples were collected, medicine class, and the number of samples tested and failed were extracted in Excel.
We used the 12-item Medicine Quality Assessment Reporting Guidelines (MEDQUARG) to evaluate the reporting standard of medicine quality studies. 16,17 Studies not included in the previous review were rated by two reviewers (H. C., Y. L.). A Spearman's correlation coefficient between reviewers was assessed for interrater reliability. Further information on MED-QUARG scoring and interrater reliability is reported in the supplemental materials.
Meta-analysis across substandard and falsified samples. Two separate meta-analyses were conducted. First, we estimated the prevalence of substandard and falsified medicines across all studies that assessed medicine quality in LMICs using a random-effects model, taking into account study sample sizes and MEDQUARG scores. A subgroup analysis was performed to illustrate the variation in the average weighted prevalence of substandard and falsified medicines across regions and therapeutic categories.
To assess the heterogeneity across studies, we evaluated the results of the random-effects model based on Cochran's Q and I 2 . Effect modifiers were assessed to identify study features that may be associated with heterogeneity across studies included in the meta-analysis. We tested five potential effect modifiers using a mixed-effects model: publication year, region, medicine category, number of samples tested, and MEDQUARG scores. A Baujat plot analysis was conducted to examine the influence of each study on pooled results. 18 A funnel plot and funnel plot asymmetry test assessed potential publication bias. 19 Additionally, we examined which studies exerted the most influence on the pooled weighted result using an influence plot analysis. 20 These results are reported in the supplemental materials.
Meta-analysis among samples that reported API levels. A second meta-analysis was conducted among studies that reported API amounts in medicine samples tested. Studies were included if they reported the percentage API of all failed samples or reported the number of samples within API ranges. Studies that reported adequate data were included whether they found any substandard or falsified medicines.
We recorded the number of failed medicine samples reported into categories of API level deviations. We documented whether failed samples were reported to contain 1) no and/or incorrect API, 2) , 50% API, or 3) , 80% API. These categories were not mutually exclusive where samples could be classified into more than one category. For example, a sample with 0% API was included in counts containing , 50% API and in the classification for , 80% API. On the other hand, a sample that reported to have , 80% API without specifying the actual API amount was only included in the , 80% API category. Medicine samples with , 80% of API are considered to be "extremely deviating" from specifications and in the absence of evidence of falsification these medicines can be considered likely substandard, 13 whereas those with , 50% of API can be considered likely falsified. 11 Where available, we categorized samples that were documented as having incorrect labeling or false packaging because this is a common sign of falsification. We also recorded when authors claimed the samples were falsified without presenting data.
We estimated the pooled prevalence of medicines with 0% API and/or incorrect API, medicines with , 50% API, and medicines with , 80% API using random-effects models weighted by sample size and MEDQUARG scores. Studies with larger samples and higher MEDQUARG scores contributed greater weight. A subgroup analysis was conducted to examine the variation in API levels across regions and therapeutic categories.
This systematic review and meta-analysis was registered in the international prospective register of systematic reviews (PROSPERO) database (#CRD42020188678). Results are reported in line with the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines.

RESULTS
Systematic review. Combined searches resulted in a total of 3,537 articles after removing duplicates, which were screened based on titles and abstracts. After conducting full-text screening of 1,043 studies, 130 studies were included in this systematic review ( Figure 1; see supplemental materials for a list of studies).
The random-effects model showed considerable heterogeneity between studies (I 2 5 99.92%) and funnel plot asymmetry showed publication bias (P , 0.001). Effect modifiers for the number of samples tested and region were significant (P , 0.05) in explaining some of the heterogeneity between studies (see supplemental materials). The heterogeneity demonstrated that studies in Africa and Asia, and those testing fewer samples tended to have higher prevalence of substandard and falsified medicines.
Among the 99 studies that included information on API levels and found medication samples that failed quality testing (9,724 samples), we found 25.9% (95% CI: 19.3-32.6%) that were reported to contain , 80% API ( Figure 3). The remainder failed other quality tests (e.g., disintegration, dissolution, degradation, presence of impurities, visual and physical inspection), contained API levels . 80% but below pharmacopeia standards or had API levels . 100%. Moreover, 13.8% (95% CI: 9.0-18.6%) of failed samples were reported to contain , 50% API, and 12.5% (95% CI: 7.8-17.3%) reported finding no or incorrect API. Figure 4 presents a subgroup analysis. The proportion of samples reported to contain , 50% API was highest in Asia at 23.4% (95% CI: 11.2-35.7%), compared with 12.7% FIGURE 2. Forest plot of overall prevalence of substandard and falsified medicines. Sample size includes all medicine quality study samples tested. Antimalarials include studies that examined antimalarials but not antibiotics. Antibiotics exclude studies that examined antimalarials. Antimalarials and antibiotics category includes studies that examined both together. Sample sizes of 1) antiretrovirals, 2) antihypertensives, 3) analgesics and anti-inflammatories, and 4) uterotonics include studies that investigated the specific therapeutic category but not antibiotics or antimalarials, and may or may not include other therapeutic categories.

DISCUSSION
Our results demonstrate that a quarter of the medicines that failed API quality tests in LMICs were reported to contain reduced API of , 80% of the stated amount. Only 12.5% of failed samples were found to have no API at all, an incorrect API, or both. This is an important finding because falsified medicines dealing with criminal activity tend to attract more attention than substandard medicines. Yet our results demonstrate that medicines with reduced API are also a pervasive problem, one that governments and policy makers need to allocate more resources toward combatting. Because both substandard and falsified medicines pose a threat to public health, it is critical to direct resources at them differently.
Our results provide some insight into where NMRAs should focus their attention. Medicines deviating from specifications with API , 80% were most commonly reported in Africa, followed by Asia. The prevalence of poor-quality analgesics and anti-inflammatories (46.8%) as well as uterotonics (46.6%), which are likely substandard, is alarming. Substandard medicines slightly deviating from standards can indicate, or are likely to arise from limited technical capacity, and deficient storage conditions at dispensing sites, where interventions should aim at ensuring sound practices. 3,121 Substandard medicines can be reduced by strengthening Good Manufacturing Practices (GMP), Good Distribution Practices (GDP), Good Storage Practices (GSP), alongside medicine registration, prequalification of suppliers, and recalls. 122,123 On the other hand, medicines reporting to contain , 50% API comprised a larger portion of poor-quality medicines in Asia (23.4%) compared with Africa (11.4%). Moreover, antimalarials and antibiotics were the therapeutic classes most likely reported to contain low API of , 50% API (18.0% and 16.7% of all poor-quality samples, respectively). This could be a sign of more falsification of these medicines, something that would require further testing and confirmation by NMRAs. Tackling falsified medicines requires coordination with law enforcement or customs authorities and may involve increased regulatory oversight, legal framework for prosecution, customs screening, post-market surveillance, and medication safety alerts. Falsification tends to flourish under high demand for medicines and poor governance,  where criminals intend to make a profit. 24,124 Therefore, preventing shortages or stock-outs and ensuring medication access are important parts of the solution. 3 According to the WHO definition, a falsified medical product is one that intends to deceive. 6 However, intention is difficult to assess. We found that most medicine quality studies do not report whether the product authenticity was confirmed by the manufacturer. Therefore, we used API amounts as a proxy to assess whether medicines are likely substandard or falsified. Although it is generally agreed upon that medicines with no or incorrect API are falsified, 11 this cutoff would miss other falsified medicines intentionally manufactured with reduced amounts of API. We reasoned that medicines with , 50% API are likely to be falsified given that there was likely to be deliberate intent to make such medicines where no confirmation of intent was provided. In the absence of ability to confirm the intent to deceive, we consider that medicines containing , 50% API without evidence of decomposition is reasonable to denote likely falsification. 11 Furthermore, we endorse the earlier call 11 to improve reporting guidelines for medicine quality studies to distinguish substandard from falsified medicines ( Table 2). Most medicines reported to be of poor-quality in LMICs did not specifically report the API amount of each sample. This makes our meta-analysis among samples reporting API levels conservative, because data were not available to classify every tested sample clearly and definitively. Currently, inconsistencies in reporting and combined results across countries, medicines, and sampled locations make it difficult to adequately assess risks and devise targeted interventions. We suggest that authors include exact API amounts rather than reporting only the number of samples that failed testing or API ranges, with further information on how and where those samples were obtained. We recommend that visual inspection, which can signal potential falsification, 125 be accompanied by chemical testing to assess API amounts, along with an attempt to communicate with the manufacturer to confirm the original source. For medications with , 80% API, we suggest that studies report whether evidence of degradation exists to differentiate between samples that had degraded after manufacture and samples that were produced with insufficient API amounts. We also suggest that results of dissolution or disintegration tests be reported alongside API results when assessing the quality of tablets. Consistent and accurate reporting of medicine quality would not only aid in comparability of results but also inform countermeasures.
There are several limitations to our analysis. First, systematic reviews are inherently limited by the search strategies used, databases searched, and inclusion and exclusion criteria applied. Our review update focused on PubMed as previous findings showed that few unique articles were identified from other databases. 9 By cross-referencing with the Medicine Quality Scientific Literature Surveyor database, we believe we have captured the most pertinent literature. Second, meta-analyses are limited by the quality of included studies and the biases they contain. 126 To minimize the impact of poor-quality studies in our analysis, we selected studies that tested 50 or more samples and weighted our meta-analyses by study sample size and MEDQUARG scores. Third, we observed considerable heterogeneity across medicine quality studies in reporting. For example, a considerable number of publications only reported API amounts below a cutoff rather than presenting a breakdown of actual API amounts of each sample. This prevented us from being able to develop mutually exclusive categories in our analysis. Our results for samples with 0% or , 50% API categories may be conservative because we were not able to assess the actual API amounts in some publications. Many publications were missing information on the criteria used to determine that a sample had failed. We suggest guidelines for reporting medicine quality studies to reduce reporting inconsistencies in the future. Lastly, our meta-analysis is likely influenced by publication bias where many studies are conducted in Africa and Asia testing antimalarials and antibiotics. Testing and reporting the quality of a wider range of medical products around the world will lend to a more comprehensive picture of the risks posed by substandard and falsified medicines. Despite these limitations, this meta-analysis offers a comprehensive and scientifically grounded method for differentiating poor-quality medicines across LMICs by reported API levels.

CONCLUSION
This study contributes to the existing literature by providing an estimate of the magnitude of the problem of substandard and falsified medicines and examining the amounts of API in medicine samples that fail quality testing. Our findings of 12.4% overall prevalence of substandard and falsified medicines are consistent with previous analyses and WHO reports. 1,9,17,[127][128][129] Our analysis goes further by finding that nearly one in seven poor-quality medicine samples were likely to be falsified based on reported API amounts of , 50%, whereas the remaining six in seven samples were likely to be substandard. Separating out substandard from falsified medicines is essential to better inform tailored interventions to ensure medicine quality throughout the supply chain. Furthermore, we propose improved guidelines for reporting medicine quality in publications to better differentiate among poor-quality medicines. Governments and policy makers should use these results to target interventions to mitigate the threats of substandard and falsified medicines.